[#259] feat: verify-quality integrated with tier gate matrix (local = CI)#378
[#259] feat: verify-quality integrated with tier gate matrix (local = CI)#378rucka wants to merge 6 commits into
Conversation
- Step 1.5 resolves risk:* tag (PR, or story card pre-publish) via shipped tier-resolve.sh; runs the tier's check set (🟢 base; +unit 🟡; +integration/E2E 🔴) - fail-safe red when untagged; missing suite => explicit "CI will fail"; matrix/KB absent => fall back to all adopted gates with notice; opt-in tiering preserved (disabled = full suite) - NO classification criteria (D18): reads tags + quality-model §4 only (single source), never the diff - conformance test (RED->GREEN) + tier-aware-gate smoke parity audit - docs: skills-catalog verify-quality row + tag-driven-gates local=CI section - regenerate .claude mirror Refs: #259 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code Review — PR #378Review InformationPR Number: #378 Review SummaryOverall Assessment
Key Changes Summary
Business Value ValidationConfirmed. Local verification now mirrors CI by reusing the same shipped Acceptance Criteria Verification
Business rules: D18 (no classification criteria in the skill) — verified: the skill composes Code Review Checklist
Security ReviewNo security surface: SKILL.md prose, docs (.mdx), a vitest content test, a bash smoke script. Illustrative Testing ReviewIndependently executed on the PR head (worktree, deps built):
The executable parity (skill resolving suites through the same Documentation Review
Detailed Review CommentsPositive Feedback
Minor Issues 💡
Nits / Questions ❓
Risk AssessmentLow. Docs/skill-definition change, no runtime production code; composition contract for Tech DebtNone material. The two Minor items above are the only debt; neither blocks. Overall DecisionAPPROVED WITH COMMENTS. All acceptance criteria and edge cases are met, tests (383 + smoke + conformance) pass on the PR head, mirror parity is exact, D18 single-source is honored, and the scope guardrail is respected. Findings are Minor/nit/question only and non-blocking. |
…age-guardrail caveat, AC4 assertion - Step 1.5: primary tag source = current-branch PR (gh pr view, no arg); story-card fallback only pre-publish. Fixes over-checking on empty $story and under-running a review-raised PR tag. - $story arg row + pre-publish edge note clarified (story-id-only semantics). - Notes: coverage guardrail is not a tier suite / not mirrored locally caveat. - AC4 conformance test pins red-verdict co-occurring with failing-command-output. - Root mirror synced (#352 guard). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…on assumption Composition Interface note: no-arg gh pr view resolves tier from the current-branch PR, so it needs the PR branch checked out; /pair-process-review composing with only $scope from off-branch degrades to fail-safe red (safe over-check, non-blocking) not the classify-phase tier. Forwarding the tier belongs to /pair-process-review (out of scope). Doc-only, both SKILL.md + mirror. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review/fix loop summary — escalating to human (non-convergence)Round 1 (commit
Quality gates round 1: PASS (knowledge-hub vitest 383/383, skill-refs 16/16, mirror guard green, check:links clean, markdownlint clean). Round 2 (commit
Quality gates round 2: PASS (skills:conformance 39/39, knowledge-hub vitest 383/383, mirror guard 15/15, check:links clean, markdownlint clean). Full detail in the working review log: Still-open findings (round 3) — flagging for human reviewTwo new Minor findings surfaced on this latest pass, both still open (no fix applied yet — escalating rather than looping further):
Two rounds of iteration have already resolved 4 findings cleanly; these last two are being surfaced for a human call on whether to spend a third fix round or accept/defer as tracked debt, rather than continuing the loop unattended. |
Code Review — PR #378Review InformationPR Number: #378 Review SummaryOverall Assessment
Key Changes Summary
Business Value ValidationDelivers the story's core value: local verification runs the same tier check set CI would run — no over-checking green work, no surprise reds — without duplicating the matrix (D18). Verified: the skill's documented matrix (🟢 install+lint+type+build; 🟡 +unit; 🔴 +integration+e2e) is byte-faithful to both Acceptance Criteria Verification
Business rules (no classification criteria / D18; adoption gate-command overrides; idempotency) and edge cases (pre-publish story-card fallback; missing suite = explicit fail; matrix/KB absent = full-suite fallback + notice) are all present in the prose. Verification Performed
Testing ReviewTests match this repo's convention for prose skills: content-invariant conformance (grep on SKILL.md) + structural parity + a smoke scenario that behaviorally executes the shared Security ReviewNo security surface. Read-only skill prose (the sole write, Step 5.C, is pre-existing and untouched). No secrets, inputs, or auth. Detailed Review CommentsPositive Feedback
Minor Issues 💡
Questions ❓
Adoption & ADR ComplianceNo new technical decision — the tier/suite logic lives in the already-shipped, smoke-tested Definition of Done
VerdictApproved with Comments. All four ACs met and independently verified against the single-source matrix; mirror clean; smoke suite green. Two Minor doc items (secret-scan caveat completeness; test-comment precision) and two Questions (DoD fixture interpretation; review-path over-check follow-up) — none blocking. Independent review via |
…-parity comment - SKILL.md (source + mirror): add "Secret scanning is NOT mirrored locally" caveat parallel to the coverage-guardrail note (D24, unconditional at every tier incl. green; owned by setup-gates/assess-security) — completes the local=CI parity carve-out list. - verify-quality-gate-matrix.test.ts: tighten header comment — smoke test asserts the skill *references* the shared helper; the helper's own resolution is behaviorally executed (#258 block). Skill prose is not itself executed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
@/private/tmp/claude-501/-Users-gianlucacarucci5-projects-pair/6b978b2a-c3b5-4d25-be2b-01366429c4a9/scratchpad/comment-259.md |
Code Review — PR #378Review InformationPR Number: #378 Review SummaryOverall Assessment
Key Changes Summary
Business Value ValidationValue ("local = CI: no surprise reds at the gate, no over-checking on green") is delivered for the tier check-set + verdict, verified structurally (same helper CI uses). Two carve-outs (coverage guardrail, secret scanning) are honestly documented as out of parity scope — see Minor #1. Code Review ChecklistFunctionality — Acceptance Criteria
Business rules & edge cases
Technical Standards / Architecture
Security Review
Testing Review
Documentation ReviewDocs site updated per DoD: Detailed Review CommentsPositive Feedback
Minor Issues 💡
Questions ❓
Risk Assessment
Definition of Done
Decision: APPROVED WITH COMMENTSNo critical or major issues. All four acceptance criteria are met and independently verified (matrix parity across three sources; 15/15 conformance + smoke pass locally). Comments are advisory (documentation branding, one under-specified interaction, a nit) — none block merge; #382 already tracks the one deferred composition-path improvement. 🤖 Independent review via |
…on no-op, repo-root CWD note - Hedge SKILL.md + skills-catalog description: 'mirrors CI for the tier check set (tier-check parity)' + name the two carve-outs (coverage-guardrail, secret-scan) so the one-liner no longer over-promises past the body. - Clarify $scope x tier narrow-only intersection: empty intersection is a no-op reporting SKIPPED (tier), never a FAIL, never a widen. - Note the Step 1.5 bash sources the helper assuming repo-root CWD (as CI runs it); the ../../../.pair link path is this file's relative path to the same asset, not a second copy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…gradation Disabled/full-suite branch now states absent suites are SKIPPED (not failed) and missing-suite-fail applies only on the enabled/tier-required path, so the branch is unambiguous read in isolation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remediation — full review/fix cycle (rounds 1–5), review cleanConsolidated response mapping every finding raised across the review/fix loop (initial review + re-reviews) to its resolution. Full round-by-round detail lives in the (now-removed) working log Findings from the initial review (this comment)
Findings surfaced during the remediation loop (rounds 2–5)
Accepted / non-actionable (with disposition)
Final verdictAll findings from the initial review and every subsequent re-review round are resolved or explicitly accepted/deferred with disposition above. The latest independent re-review of the current head ( Review clean. Ready for the human merge decision. |
Summary
What Changed
pair-capability-verify-quality(SKILL.md) now resolves the item/PRrisk:*classification tag and runs locally exactly the check set the CI tier-aware gate (#258) would run for that tier — new Step 1.5: Resolve the Tier Gate Matrix (CI parity), tier-scoped Test gate (Step 4), and Tier + Check set surfaced in the output report. v0.4.1 → 0.5.0.quality-model.md §4/ the KB gate matrix delivered by Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258 — sametier-resolve.shhelper the CI pipeline uses, so local = CI by construction, not by re-implementation.Also: docs site
verify-qualityskills-catalog row rewritten with tier behavior, and a new "Checking locally before you push (local = CI)" section ontag-driven-gates.mdx.Why This Change
Closes story #259 of epic #210 (CI/CD gates tag-driven). Without this,
verify-quality(used standalone, or composed by/pair-process-implementand/pair-process-review) can pass locally green while CI still runs a wider tier check set (or vice versa) — surprise reds at the gate, or wasted local effort over-checking green work. This wires the same gate matrix CI already reads (#258, merged) into the local capability, so local verification mirrors CI exactly.Story Context
User Story: As a developer, I want
pair-capability-verify-qualityintegrated with the tier gate matrix — resolving the item/PR classification tags and running locally exactly the checks the CI gate will run for that tier — so that local verification mirrors CI: no surprise reds at the gate, no over-checking on green work.Acceptance Criteria: AC1–AC4, all covered by the new conformance test
verify-quality-gate-matrix.test.ts(15 cases) + the extendedtier-aware-gate.shsmoke scenario:risk:green→ install+lint+type+build only, same set as CI) — covered.risk:yellow/risk:redwidens per matrix: unit from 🟡, integration+E2E on 🔴) — covered.Changes Made
Implementation Details
verify-quality— resolves tier (PR tag → story tag → fail-safe red) and the required check set viatier-resolve.sh(D18: single matrix source, no criteria duplicated in the skill).verify-quality-gate-matrix.test.ts, 15 cases, written RED→GREEN) pinning the AC invariants + matrix parity withtier-resolve.sh; extendedtier-aware-gate.shsmoke scenario with a verify-quality local=CI parity audit (same helper CI uses).Files Changed
packages/knowledge-hub/dataset/.skills/capability/verify-quality/SKILL.md(source of record).claude/skills/pair-capability-verify-quality/SKILL.md(regenerated mirror — deterministic name-prefix transform only, same approach as PR [#314] feat: wire template-override resolution into skills #367)apps/website/content/docs/reference/skills-catalog.mdxapps/website/content/docs/concepts/tag-driven-gates.mdxscripts/smoke-tests/scenarios/tier-aware-gate.shpackages/knowledge-hub/src/conformance/verify-quality-gate-matrix.test.tsNo deployable service/package changed — this is a KB skill definition, its
.claudedistribution mirror, docs, a conformance test, and a smoke-test extension. Services to Release section omitted per the conditional rule.Testing
Test Results
Note: knowledge-hub
prettier:checkreports a pre-existing drift insrc/conformance/refine-story-draft-to-ready.test.ts(NOT touched by this story, unrelated to this diff). Files touched by this PR are prettier-clean.Testing Strategy
tier-resolve.sh's own matrix output.tier-resolve.shhelper CI uses, proving local = CI is structural (shared code path), not a documentation claim.Reviewer Guide
Review Focus Areas
verify-qualitystill contains zero classification/matrix criteria of its own; it composestier-resolve.sh(delivered by Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258, merged) rather than re-implementing the matrix. Grep-verifiable, pinned by the smoke scenario./pair-process-reviewskill (explicit constraint on this story); confirm the diff has none of that..claudemirror parity — the mirror diff should be name-token-only vs. the dataset source (verify-quality→pair-capability-verify-quality,/implement→/pair-process-implement,/review→/pair-process-review,/classify→/pair-capability-classify); no independent drift.tier-resolve.sh(Tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #258); this story composes it. Confirm this doesn't warrant a new production module per the gate-tooling ADL (2026-07-13-gate-tooling-code-in-tested-modules.md).Testing the Changes
git checkout feature/US-259-verify-quality-gate-matrix pnpm install pnpm --filter @pair/knowledge-hub test pnpm --filter @pair/knowledge-hub run skills:conformance bash scripts/smoke-tests/scenarios/tier-aware-gate.shDependencies & Related Work
tier-resolve.shhelper, PR [#258] feat: tier-aware pre-merge pipeline — reads classification tags only, fail-safe red #362) — merged, present onorigin/main.apps/pair-cli) updated where install/update/distribution is touched" — N/A here. No new distributed asset:verify-quality's SKILL.md andtier-resolve.shalready ship; install/update/distribution machinery is untouched by this diff.risk:*label at time of PR — opened without tags per the propagation rule (copy-only; authoritative classification happens in review).